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I. Introduction 


1 Iam a professor of English Language and Linguistics at Brigham Young University, Utah and have been at the university 
since 1988. From 2007 to 2013, I was chair of the Linguistics and English Language Department, one of the largest linguistics 
department in the U.S. with respect to undergraduate enrollments. The Department consists of 24 full-time faculty. [am currently 
Ludwig-Weber-Seibach Humanities Professor within the College of Humanities. This is a five-year professorship given in 
recognition of one's contributions to the university and to the general field. Throughout my university-level teaching career, 
I have taught a range of graduate and undergraduate classes including general linguistics, sociolinguistics, psycholinguistics, 
varieties of English, grammar and usage, semantics, discourse analysis, second language acquisition, TESL (Teaching English 
as a Second Language) methodology and applied linguistics research methodologies. 


2 Over the past 10 years, I have taught a specialized class in Forensic Linguistics; I have presented papers in forensic linguistics 
at national and international conferences; and I have published in this area. Most recently, I co-authored two peer-reviewed 
papers in the Harvard Latino Law Review, the first of these papers was co-authored with Judge Lynn. W. Davis of Utah's Fourth 
District Court. 


3 I received my M.A. and Ph.D. in Linguistics from the University of Southern California with a sub-specialization in 
Sociolinguistics- the study of how language functions at the social level. My Ph.D. dissertation consisted of a study of minority 
language behavior and behavior toward language within a Los Angeles area school district. 


4 My attached curriculum vitae (Exhibit A) reveals a research and publications record and involvement in other scholarly 
activities in the applied linguistics field that qualifies me to provide expert analysis of legal and non-legal issues involving 
English as a second language proficiency, non-standard Englishes, discourse analysis, communication breakdowns, and textual 
analysis. As such, I have been involved as a consultant and designated as an expert witness in cases involving linguistic analysis. 
I have testified numerous times in U.S. state and federal courts. A brief synopsis of these cases is provided below. 
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5 lama member of the International Association of Forensic Linguistics (“IAFL”), the premier forensic linguistics association. 
As such, I adhere to the IAFL's code of practice. This code of practice recommends inserting the following statement in each 
report. 


This report is based on my professional knowledge and expertise, and on my research using established and accepted linguistic 
knowledge and methodology. The data and sources that I considered in forming the professional opinions expressed here are 
referenced where relevant throughout the report. If sworn as a witness, I could testify competently to the matters stated herein. 
I understand that my duty in providing written reports and giving evidence is to assist the justice system- and that this duty 
overrides any obligation to the party by whom I am engaged or the person who has paid or is liable to pay me. I confirm that 
I have complied and will continue to comply with my duty. I am being compensated in this case at an hourly rate of $350.00. 
My compensation is not contingent in any way on the outcome of this case. 


6 It should be noted that, to a large extent, as I will explain below, the motivation for the IAFL for developing this code of 
conduct came from discussions involving the reliability and validity of forensic stylistics, the research method used in Dr. 
McMenamin's Dutcher v. Bold report. 

7 With respect to the expertise required in this present case, hereafter designated as the “Dutcher v. Bold Case,” I provided 
authorial attribution testimony at an evidentiary hearing concerning the writer of two anonymous threat letters (see United States 
v. Zajac, attached as Exhibit B). In this case, the testimony offered by the prosecution witness was based on “forensic stylistics,” 


the analytical method used by Dr. Gerald McMenamin in Dutcher v. Bold. 


8 I was recently contacted by Cheylynn Hayman and David Reymann of Parr, Brown, Gee and Loveless, Salt Lake City, Utah, 
and asked to undertake the following: 


a. Respond to a report written by Dr. Gerald McMenamin where he conducted “a review and analysis of the Questioned and 
Known screenplays in the matter of Dutcher v. Bold.” 


b. Write a report based upon my findings. 


II. Research Design 


9 In accordance with this request, I determined to conduct a multifaceted research design that involved the following: 
a. Conducting a review of forensic linguistics published, peer-reviewed authorial attribution research. 

b. Conducting a detailed analysis of Dr. McMenamin's report. 

c. Conducting an assessment and of that report with respect to the reliability and validity of its findings. 

d. Offering an opinion of the reliability and validity of that report and its findings 


e. Writing a report detailing the rationale for that opinion. 
Ii. Opinion 


10 After conducting a study as outlined above and as described in detail in the following paragraphs, I have determined the 
following with respect to the scientific reliability and validity of Dr. McMenamin's Dutcher v. Bold opinion: 
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a. Dr. McMenamin's “forensic stylistics” approach fails the test of scientific reliability. Dr. McMenamin's approach consists of 
comparing stylistic “markers” that he identifies in known and questioned texts. But none of these so-called markers selected by 
Dr. McMenamin have been subjected to any prior scientific or empirical testing and none of them has been proven to reliably 
predict authorship. Indeed, the markers used in Dr. McMenamin's approach are selected on an ad hoc and impressionistic basis 
without any grounding in empirical, scientific research. 


b. Dr. McMenamin's method consisted of applying “forensic stylistics” to an analysis of two known screenplays written by 
Dan Gilroy and a purported questioned or “unknown” Nightcrawler screenplay claimed to be written by Mr. Gilroy. He opines 
that, “it is highly probable that Dan Gilroy is not the author of the QUESTIONED Nightcrawler screenplay.” Dr. McMenamin 
improperly expresses his conclusion in terms of “probab [ility]” when, in fact, he has offered no statistical evidence to support 
the notion that any of the markers he has selected makes authorship attribution more or less probable. Thus, Dr. McMenamin's 
method fails the test of scientific reliability because, in the absence of prior empirical testing and verification of his results, Dr. 
McMenamin cannot demonstrate that his approach produces stable and consistent results. 


c. Dr. McMenamin claims to have discovered linguistic features that show that Dan Gilroy's screenplay writing style, as 
determined by an examination of selected linguistic features in the screenplays known to have been written by Gilroy, do not 
appear in the questioned Nightcrawler screenplay. However, as noted above, there is no scientific evidence that these features 
are predictors, or markers, of an individual's writing style that can predictably and reliably identify authorship. The presence 
or absence of Dr. McMenamin's features or markers has not been established in the research literature as having any predictive 
value. Nor has he demonstrated that the comparison of the two texts in question represents a sufficiently large sample size 
for the performance of forensic authorship analysis. Thus, Dr. McMenamin's research method and his derived opinion fail the 
validity test in that they do not measure what they claim to measure. 


d. Having failed both the reliability and validity tests, it is impossible to ascribe any scientific worth or credibility to Dr. 
McMenamin's opinions with respect to the author of the Nightcrawler screenplay. They appear to be the result of advocacy 
rather than a representation of scientific principles. These failings of the forensic stylistics approach are well recognized in the 
field of forensic linguistics at large. 


IV. Rationale for my Opinion 


11 The following discussion will provide evidence of the assertions contained in the opinion above. I will first briefly review 
the field of authorial attribution within general forensic linguistics concluding with a critique of the research methodology 
used by Dr. McMenamin. I will then evaluate Dr. McMenamin's findings in terms of reliability and validity by conducting an 
approach to authorship attribution demonstrating that Dr. McMenamin's approach is highly malleable. A summary of findings 
and rationale for my opinion will then conclude this report. 


V. Authorial Attribution and Forensic Stylistics 


12 The forensic linguistics subfield of “Authorial Attribution” has a substantial literature (see “References” at the end of this 
report). Much of this literature focuses on ensuring that strict scientific standards are met. In order for any method of Authorial 
Attribution to be accepted by the forensic linguistics community, it must be shown to produce reliable and predictable results- 
results that have been subjected to rigorous empirical testing. For example, some forensic linguists take a “big data” quantitative 
approach to determining authorship. In essence, these approaches use computational linguistic methods to determine “a priori” 
the linguistic features of an “idiolect” (an individual's “wordprint’’), and then apply that knowledge to discovering the authorship 
of unknown texts. However, regardless of the amount of data that is examined, the linguistic features examined in any Authorial 
Attribution must be shown to produce reliable results. 
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13 A reliable research method is replicable in the sense that an independent researcher should be able to replicate the same study 
and determine the same results. However, in forensic stylistics, as practiced by Dr. McMenamin generally and in his Dutcher 
v. Bold report, the linguistic features, or style markers, he chooses to use to form his conclusions are chosen intuitively and 
they are not based upon any research foundations outside of his own ad hoc experience. Consequently, another researcher can 
examine the same KNOWN and QUESTIONED data-set and intuitively select a totally different set of style markers leading 
to a potential different research result. Such a process is not scientifically reliable. 


14 A valid research method in linguistics bases conclusions on a set of linguistic features that have been independently shown 
to correlate to, or predict, certain outcomes. For example, a researcher may observe that secondary school students with college 
aspirations adhere more closely to features of standard American English. The researcher develops a hypothesis and conducts a 
review of related research literature. In that literature review, the researcher may discover that pronouncing -ing ending present 
participles such as running with a /g/ is an indicator of social mainstreaming or convergence whereas pronouncing runnin’ 
with /g-dropping/ indicates social divergence from the mainstream. The researcher then uses that linguistic feature to test the 
hypothesis. In so doing, valid linguistic markers that have been established in peer-reviewed literature by independent research 


have been used in the research design. 


15 Dr. McMenamin's linguistic markers have generally not been established in the literature as markers of individual authorial 
style. In his Dutcher v. Bold report, the presence or absence of a particular linguistic feature, or the frequency of a particular 
feature may measure normal individual variation. For example, I may say or write something one way one day, and say or 
write the same meaning another day totally differently. In that sense, the variability between the KNOWN and QUESTIONED 
screenplays may be ascribed to “intra-textual” variation: basically the normal variation we all use in language production. 


16 This is especially the case when the KNOWN and QUESTIONED samples are separated over significant time. For example, 
I often have my senior or graduate students conduct a linguistic analysis comparing linguistic features in their freshmen class 
term papers with those in their senior class terms papers. They find differences in their vocabulary, their sentence structure, 
their formatting and in the logical development of the essays. These differences in linguistic markers, or style markers, can 
be ascribed to normal variation over time, which is what the established literature would declare. Unfortunately, using the 
theoretical assumptions of Dr. McMenamin's flawed forensic stylistics method, the differences could be classified as style 
markers indicating different authorship-a totally incorrect conclusion. In this sense, then, Dr. McMenamin's style markers lack 
scientific validity because they measure something that is incorrect. 


17 Proponents of “forensic stylistics,” rely on a case-by-case, ad hoc and impressionistic selection of stylistic markers when 
opining on issues of Authorial Attribution. These stylistic markers typically have not been subjected to any empirical testing 
to demonstrate their predictive value. 


18 This is the approach employed by Dr. McMenamin in the present case. Proponents of forensic stylistics claim that that each 
person has an individual style of writing which is revealed through various “style markers”. These style markers are chosen on 
a case-by-case basis, because, according to Dr. McMenamin (2002) “authorship identification requires the identification of an 
aggregate of markers, each of which may be found in other writers, but all of which would unlikely be present together in any 
other writer.” Even if this were true, we would have no reason to believe that the ad hoc selection of style markers that have not 
be subjected to any empirical testing can reliably predict authorship. Dr. McMenamin has offered no such empirical evidence 
to support his selection of the style markers identified in his report. 


19 This failing of Dr. McMenamin's forensic stylistics approach to empirically support its claims is well-known in the forensic 
linguistics community. Commenting on Dr. McMenamin's approach to forensic stylistics, Dr. Carole Chaski, the Executive 


Director of the Institute for Linguistic Evidence has observed: 


As actually practiced in the reports ..., [Dr. McMenamin's] method consists of two steps: 
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1. Select style markers by reading the questioned (“Q”) and known (“K”’) documents; 


2. Decide the authorship of the questioned document(s) based on the style markers by listing similarities and/or differences and 
deciding which similarities and which differences are important or not. 


The method offers: 
i. no protocol for the order of reading Q or K first, or back and forth between Q and K, 


ii. no protocol for internal consistency testing of K or Q documents, so that any number of Q documents can be put together, 
in violation of a standard forensic science principle of non-contamination; 


iii. no protocol for determining the importance or “significance” of stylemarkers, 
iv. no use of statistical analysis (in actual case reports); 
v. no standard reference set of style markers to be reviewed in each case. 


20 Dr. Chaski (2013) continues: “Number (v) is especially important because it means that the method allows the examiner to 
pick and choose style markers without any predictability. This fundamental methodological flaw enables a host of problems, 
all rooted in subjectivity. On the one hand, it is essentially impossible to replicate a forensic stylistics analysis, while on the 
other hand, it is always possible to find an alternative analysis and opposing conclusion. This is the dilemma of any “pick and 
choose’ method.” (emphasis added). 


21 Dr. Chaski also quotes Dr. Geoffrey Nunberg, linguistic and professor at the Berkeley School of Informaiton, who stated: 
“Professor McMenamin's methods are not based on well-established theoretical principles nor are they consistent with rigorous 
practice in the statistical analysis of written texts. McMenamin has performed no statistical research that would give any 
scientific grounding to his conclusions. I would not classify McMenamin's work as bad science; rather, it is not science at all.” 


22 As in the present case, Dr. Nunberg observed that “Professor McMenamin's choice of the features used in document 
comparison is arbitrary and subjective, and unmotivated by any empirical research; another set of features could well have been 
chosen that would have given very different results. His method could not pass the test of independent replicability.” Finally, 
Dr. Nunberg observes that “ [i]n the absence of a prior statistical analysis, McMenamin has no scientific basis for distinguishing 
those features of a document that are likely to be cues of authorship, nor does he have any grounds for assuming that the 
appearance of the same feature ... in two texts offers significant evidence of common authorship ... Scientifically speaking, 
McMenamin's analyses are worthless.” 


23 Preeminent linguist and language commentator, Dr. David Crystal (1995) expressed similar concerns in his review of Dr. 
McMenamin's book on forensic stylistics for the prominent peer-reviewed linguistics journal Language. Crystal stated that Dr. 
McMenamin “makes a lot of the scientific basis of the proposed subject;” however, “[t]here isn't a single test of statistical 
significance in the book, and the whole of the opening chapter, which is the central illustration of the approach, is based on 
impressionistic statements of varying levels of vagueness.” Dr. Crystal also observes that, as in the present case, Dr. McMenamin 
“talks in a semistatistical way ... but he does not present the statistical analysis which would make such comparisons convincing. 
Indeed, at several points, one wonders whether it would in principle be possible to do so, given the sample sizes, and the lack of 
lexical frequency norms.” Dr. Crystal continues, stating that “[u]nfortunately, there is an enormous gap between this perspective 
and the actual approach [Dr. McMenamin] uses, both in theory and in practice. The problem is that, after reading this book, 
lawyers might be forgiven for thinking that this is an orthodox account of a domain of applied stylistics. It is not.” 
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24 The same concerns expressed above with respect to Dr. McMenamin's approach are present in Dr. McMenamin's report in 
Dutcher v. Bold. Dr. McMenamin has impressionistically selected a collection of style markers that have not been subjected to 
empirical testing and have not been shown to reliably indicate authorship. Even if we were to accept the predictive value of Dr. 
McMenamin's style markers, Dr. McMenamin has presented no evidence that the size of the sample he employs is sufficient to 
draw reliable conclusions about authorship. Professor Lawrence M. Solan (2013, p. 574), head of the Center for the Study of 
Law, Language, and Cognition at the Brooklyn Law School has stated that “it will be incumbent upon those whose work is more 
intuitively stylistic to demonstrate its scientific underpinnings.” Dr. McMenamin has failed to do so. It is this methodological 
weakness, rather than its theoretical assumptions, that is at the heart of my misgivings regarding the reliability and validity of 
forensic stylistics as it is currently practiced, and as demonstrated by Dr. McMenamin's analysis in Dutcher v. Bold. 


25 These concerns regarding the unscientific methodological weaknesses of forensic stylistics gained public prominence in 
2011 when Dr. McMenamin submitted a report in a case involving Facebook founder, Mark Zukerberg, as to who wrote a 


set of disputed e-mails. The case was reviewed in the New York Times by linguist and language columnist Ben Zimmer. ! 


Zimmer references a statement by Dr. Ron Butters, a highly respected linguist and then out-going president of the International 
Association of Forensic Linguists (“IAFL”). Zimmer states: 


But Mr. McMenamin's report has raised eyebrows in the forensic linguistics community. Earlier this month, the outgoing 
president of the International Association of Forensic Linguists, Ronald R. Butters, publicly questioned whether Mr. 
McMenamin could actually establish that Mr. Zuckerberg likely did not write the e-mails based on such slender evidence. For 
example, the would-be Zuckerberg e-mails had one instance of uncapitalized “internet,” while a sample of e-mails known to be 
sent by Mr. Zuckerberg had two capitalized instances of “Internet.” “Are we really doing ‘scientific’ and ‘linguistic’ analysis 
at all when we simply note instances or absences of this or that superficial textual feature?” Mr. Butters asked. 


26 Indeed, in his remarks before the IAFL on the occasion of his retirement, Dr. Butters (2011) specifically singled Dr. 
McMenamin's opinion in the Facebook case as presenting “data of dubious value [that] can mislead [a jury]” and as illustrative 
of the need for the IAFL to adopt a set of standards and practices for consulting work on Authorial Attribution cases. He states 


[O]ne-third of the ‘unique set of variables' upon which McMenamin (2011) bases his firm conclusion that the Q and K authors 
are very likely different are composed of exactly 18 reported tokens. A similarly small number of features make up the other 
members of the unique set. It seems reasonable to at least begin to question the scientific validity of a methodology that bases 
definitive conclusions on so little data--conclusions, moreover, that are intended to be probative in litigation that may involve 
hundreds of millions of dollars and in criminal proceedings where a death penalty may be at stake. 


27 The notoriety of this case prompted a discussion on the premier academic linguistics blog “Language Log,” which is managed 
and curated by linguists at the Linguistic Data Consortium (“LDC”) at the University of Pennsylvania. In the comments to the 


post discussing the case, > several prominent linguists expressed concerns with Dr. McMenamin's forensic stylistic methodology 
similar to those highlighted above. For example, Professor Ron Butters observed that “it might be possible for forensic linguists 
to set some sort of standards for reliability and methodology, standards that, I would argue, McMenamin's report does not meet.” 


28 Professor Lawrence Solan, also mentioned above, stated: “Let us assume that McMenamin is right: The same person did not 
write both the known and questioned emails. The problem at this point is that the analysis does not appear from the report to be 
based on methodology developed through research in which ground truth concerning authorship is known in advance and the 
likelihood of the method yielding the correct result is determined. Rather, it appears to be based on a common sense intuition, 
which may indeed be right. But without the research backing up the method, we cannot be sure.” (emphasis added). 


29 In the same comments, Dr. Carole Chaski stated: “Statistically, the intra-writer variation is greater than the inter-writer 


variation, for the kinds of features forensic stylistics uses, which is why real linguists don't analyze at these levels of linguistic 
structure anyway.” 
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30 Finally, forensic expert, Marcel B. Maatley, stated that: Several years ago I was consulted on a case in Oregon. Defendant 
was convicted of murdering his wife solely on stylistic evidence, what I call “guilt by grammar.” Dr. McMenamin and another 
stylistic expert persuaded the judge that defendant had typed a photocopied document sent to Oregon State Police describing 
the woman's death. They concluded that every fact in the letter exonerating defendant was a lie, that every statement offering 
an instigative lead was a clever ruse, and that everything that said she was killed incriminated him. Dr. McMenamin relied on 
69 markers. In a murder case at the same time in San Diego County, CA, Dr. McMenamin used a different set of markers to 
prove defendant wrote an incriminating note. If the two sets of markers had been switched between the two cases, the stylistic 
evidence would have proven each defendant had not written the note in his own case but had written the note in the other case. 


In his two books Dr. McMenamin says markers are decided on a case-by-case basis. He gives no objective, or even subjective, 
guidelines for deciding what is specific to one case vis-a-vis another. Surveying thirteen cases reported in transcripts of his 
testimony, in his presentations, or in his books, I found each case had some significant difference in theory, method and 
observations when compared to any other of the thirteen cases. 


31 It is not my usual practice to cite internet blog commentary. I do so here, however, to show that the general forensic linguistics 
field has serious problems with the scientific reliability and validity of Dr. McMenamin's forensic stylistics approach. I share 
these concerns particularly with respect to Dutcher v. Bold. 


VI. Forensic Stylistics Applications to Dutcher v. Bold 


32 In his report, Dr. McMenamin opines that “It is highly probable that Dan Gilroy is not the author of the QUESTIONED 
Nightcrawlers screenplay.” Dr. McMenamin relied on his forensic stylistics method to come to this opinion. In the absence of 
any specific disclosure of Dr. McMenamin's research method, I assume he arbitrarily decided on comparing the Nightcrawler 
screenplay with two known screenplays written by Dan Gilroy: Adults Corrupt the Kids (3 pp.) and Two for the Money (109 
pp.). He provides no rationale as to why he chose these screenplays, and does not explain how he knows that they were actually 
written solely by Dan Gilroy, or if they were untouched by editors, or if they were written using a specific word-processing or 
screenplay software program. He simply assumes that these two screenplays will form the “KNOWN?” basis of his analysis, 
and create the foundation for defining Dan Gilroy's “style markers”. 


33 In so doing, he disregards the substantial sociolinguistic variation research that indicates that adult speakers of any language 
have a repertoire of styles available to them that vary according to cultural and situational contexts. I often engage my students in 
an exercise where we transform, informal spoken English, “I looked at the ball” into academic English, “Personal observations 
were conducted vis-a-vis a designated spherical object.” In so doing, we replace all the Germanic vocabulary with Latinate and 
French vocabulary (except for “a’’), we nominalize the verb (observe to observations), and we transform a simple active voice 
sentence into a complex passive sentence. 


34 Using large corpus data, Biber (1988) has shown that we regularly and subconsciously access a wide selection of linguistic 
features in order to change our language style along a spectrum ranging from informal phone conversations to formal academic 
or legal papers. This ability for each of us to engage in sophisticated “intra-textual variation” presents huge problems for “inter- 
textual variation” studies involving authorial attribution that rely on the notion of designated and static “style markers.” 


35 Indeed, such intra-textual variation may be amplified when comparing a work of fiction like a screenplay, in which the author 
is attempting to capture the voice of different characters or the tone and mood of different fictional settings and circumstances. 


36 Intra-textual variation exists even in McMenamin's Dutcher v. Bold report. He refers to the QUESTIONED screenplay in the 
singular “Nightcrawler” five times, and in the plural “Nightcrawlers” seven times. This variation is understandable, given that 
the screenplay is titled “Nightcrawlers” while the final movie title was “Nightcrawler”. Similarly, many of the ‘style markers” 
that McMenamin uses in his Dutcher v. Bold analysis, can be simply ascribed to personal variation caused by some internal 
or external experience. 
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37 After arbitrarily designating KNOWN screenplays, McMenamin then compares and contrasts the 3 pages of KNOWN Adults 
Corrupt Kids and the first 10 pages of KNOWN Two for the Money with the first 10 pages of the so-called “QUESTIONED” 
Nightcrawler screenplay. 


38 Dr. McMenamin does not provide any justification for limiting his dataset of KNOWN Gilroy screenplays to just two 
samples. Nor does he provide any justification for his decision to examine only a total of 13 pages of the KNOWN screenplay 
and 10 pages of the ostensibly QUESTIONED screenplay. We have no reason to credit (and every reason to be skeptical) of Dr. 
McMenamin's implicit conclusion that 13 pages of text is a sufficient sample size with which to make reliable determinations 
about authorship. 


39 This is especially problematic when we consider the research conducted by Grant (2007) where he attempted to determine 
the point at which authorial attribution can be determined with respect to the number of available samples. He states: 


The worked example starts with twenty comparison texts for each of three potential authors and then uses a progressively smaller 
comparison corpus, reducing to fifteen, ten, five and finally three texts per author. This worked example demonstrates how 
reducing the amount of data affects the way conclusions can be drawn. With greater numbers of reference texts, quantified and 
safe attributions are shown to be possible, but as the number of reference texts reduces the analysis shows how the conclusion 
which should be reached is that no attribution can be made. The testing process at no point results in instances of a misattribution. 


40 Grant shows that it is very difficult to make an authorial determination under five samples per author, and definitely no 
determination can be made under three samples per author. In Dutcher v. Bold, Dr. McMenamin uses two KNOWN samples 
and one QUESTONED sample. Larger sample sizes are required to account for intra-textual variability. 


41 Another curious element is noted with respect to how Dr. McMenamin further reduced his sample size. In his Dutcher v. 
Bold report, he states: 


I then examined the linguistic variation present in all writings. After observing that most stylistic variation occurred in the so- 
called “action lines” of the scripts, I extracted all action lines from the first 10 pages of QUESTIONED-WNightcrawlers, and 
from the KNOWN-Gilroy scripts: the three pages of Adults Corrupt the Kids and the first 10 pages of Two for the Money. 
(McMenamin Report, Section 8, p. 3) 


42 He then relied on an analysis of these actions lines to isolate his style markers and form his opinion. In essence, he is looking 
for variation to confirm his different author hypothesis. He selects “action lines” because he see that it is in the action lines 
where there is the greatest variation. This is further evidence of cherry picking the data within an advocacy research paradigm. 
In addition, in so doing, he vastly decreases the sample size of the writings that he is examining. 


43 Dr. McMenamin then begins his data analysis. We can assume that he looks for similarities and differences among the 
known writing samples, and searches for differences in the questioned sample. On the other hand, perhaps he proceeds from 
the questioned to the known, or he uses both processes. Regardless, he eventually develops a list of style markers that support 
his hypothesis, which, in this case is that the Nightcrawler screenplay could not have been written by Dan Gilroy. In essence, 
he cherry picks the data to confirm his hypothesis while ignoring or minimalizing data that may reject the hypothesis. 


44 At no point in his research design does Dr. McMenamin contemplate using standard qualitative-control measures to reduce 
researcher bias. Iam the Linguistics MA coordinator for my department. As such, I regularly advise Master's thesis students. If 
a student presented such a research design to me, I would immediately advise/demand that the student use an independent rater 
to replicate the analysis, and then conduct a “rater-reliability” comparison to ensure that any findings have been independently 
verified and empirically determined. I would go so far as to suggest that if any graduate-level student were to present a research 
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design as presented in Dutcher v. Bold that is so vulnerable to researcher bias as a completed Master's level thesis in linguistics, 
that student would fail the thesis defense, and the thesis would be rejected. 


45 While focusing on “dialog lines” (McMenamin Report Section 11), Dr. McMenamin acknowledges that, ... there are 
significant similarities in the dialog lines of the QUESTONED-Nightcrawler and KNOWN-Dan Gilroy scripts: 


a. The use of three suspension points [ ... ] to divide sections of dialog 

b. The use of a double hyphen [ -- ] before and after sections of dialog 

c. The use of (parentheticals) just below the character's name or within the dialog 

46 However, because he is advocating for a “different author hypothesis,” he basically ignores the consequence of these 


“significant” (his words) findings, and proceeds to list some minute differences which can easily be explained as personal 
variation in the same way that McMenamin's singular “Nightcrawler” versus plural “Nightcrawlers” is accounted for. 


VII. Thought Experiment #1 
47 Asa thought experiment, let us imagine that I have been asked to show that the KNOWN “Two for the Money” (i.e., definitely 
written by Dan Gilroy) and QUESTIONED “Nightcrawler” screenplays were both written by Dan Gilroy. Using McMenamin's 
forensic stylistics method, I can create a list of “Style marker” similarities such as McMenamin lists above: 
¢ The use of three suspension points [ ... ] to divide sections of dialog 
¢ The use of a double hyphen [ -- ] before and after sections of dialog 
¢ The use of (parentheticals) just below the character's name or within the dialog 
It is quite an easy task to cherry pick the data to find similar style markers even though Dr. McMenamin and I are unaware of 
any research that suggests that his and my selected style markers are valid and reliable indicators of same authorship. “With 
respect to word-choice, I proceed to scan both documents specifically looking for similar word or phrase choices that may be 
found in both screenplays. In essence, I'm cherry picking the data. I find the following: 
From KNOWN Two for the Money 
Course we can. Who am I gonna be? 
You're gonna be fine. Hold on! 
Hey - we're gonna be advising somewhere in the neighborhood of 20 million dollars this week. 
So listen, my office is renting out a loft this weekend, really fun group, it's gonna be a big blow-out, a PR thing - music, open bar. 


That game's gonna be won by coaching, Stu. From QUESTIONED Nightcrawler Gonna be a game changer. 


But it also means twice the sales ‘cause we're gonna be first at the scene. I'm gonna be tag-teaming every call. Your lead's gonna 
be at Motor and Washington. 
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According to McMenamin's style marker criteria, the use of these almost similar “gonna be” constructions is sufficient to support 
a claim that the two screenplays were written by Dan Gilroy. 


48 In addition, let us imagine that I am aware that recent computational work on authorial attribution concludes that one of the 
most reliable and valid indicators of same-authorship revolves around word chunks or word clusters, especially in the use of 
unusual and multiword phraseology, or n-grams. As Lancashire states: 


An author's signature can be found in his repeated clusters, the networks of repeating fixed phrases and collocations in a text. 
These clusters are fragmentary realizations of schemata in his long-term memory. Just as we recognize a face by its combinations 
of visual features, or a signature by its stroke assemblages, so we can recognize Shakespeare's writing - in the way he believed 
we do - by its repeating verbal clusters. (Lancashire, 1997, p. 137) 


49 Consequently, I scan my two screenplays for unusual phrases beginning with the KNOWN “Two for the Money” screenplay. 
I find an unusual phrase “get out of your head”. I then conduct a search of the QUESTIONED “Nightcrawler” screenplay. I 
find exactly the same phrase. In addition, the similarities continue into the following phrase with “It's a bad neighborhood.” 
Thus, from both screenplays I find: 


TONI: Get out of your head, Walter. It's a bad neighborhood. (Two for the Money Screenplay p. 23, emphasis added) and 
LOU: Get out of your head, Rick. It's a bad neighborhood. (Nightcrawler Screenplay p. 33, emphasis added) 


50 We might take this as an exceptionally strong style marker that indicates that the KNOWN and QUESTIONED screenplays 
were written by Dan Gilroy. Finally, in order to determine how common the “Get out of your head... It's a bad neighborhood” 
ten-word cluster or n-gram is, I conduct a Google web-search for the expression. To my surprise, I find that all the direct hits link 
to Dan Gilroy and Nightcrawler. Of particular relevance is an interview with Dan Gilroy conducted 31 October 2014 and found 
at: http://www.slashfilm.com/dan-gilroy-interview/. The interviewer, Russ Fischer, asks Mr. Gilroy about a memorable line: 


Russ Fischer: One line really stuck out for me: Lou telling Rick “get out of your head, it's a bad neighborhood.” It sticks in 
part because I project it onto Lou as well. 


Dan Gilroy: Yeah, it is a bad neighborhood in Lou's head. 
Russ Fischer: That line seems so specific; does it come from a particular place? 


Dan Gilroy: It's very funny you ask about that line! That line came from a friend of mine who's a producer, a gentleman named 
Jon Peters (Superman Lives producer), and I worked with him a number of years ago. Jon used to say that sometimes. “Get 
out of your head, it's a bad neighborhood!” And it always stuck with me. I loved the line, so I'll credit Jon Peters with that 
line. And proudly so, I love the guy. 


51 In summary, my thought experiment seems to have unearthed a significant word/phrase choice style marker with an 
independent provenance that links the KNOWN and QUESTIONED screenplays to Dan Gilroy. When combined with the other 
stylistic markers mentioned above, it appears that the similarities between the KNOWN and QUESTIONED screenplays are so 
strong that, using Dr. McMenamin's rating criteria, I could attest that it is “highly probable” that Dan Gilroy is the author of the 
QUESTIONED Nightcrawler screenplay. While I am offering no opinion as to the authorship of the screenplays referenced in 
Dr. McMenamin's report, the thought experiment above illustrates just how simple it is to make an ad hoc selection of stylistic 
markers to demonstrate that two texts had the same (or different) authors. 


VII. Thought Experiment #2 
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52 In the previous thought experiment, I replicated Dr. McMenamin's Forensic Stylistics method by applying it to the KNOWN 
and QUESTIONED screenplays in his Dutcher v. Bold report. This present thought experiment conducts a similar analysis of 
two of Dr. McMenamin's reports presented in other cases to show that Dr. McMenamin could not have written one of the reports 
he claims he wrote. Let us posit the KNOWN report as: Declaration of Gerald R. McMenamin in CHEVRON CORPORATION 
v. Steven DONZIGER, et al. written June 30, 2011, and presented as Exhibit 5. Let us posit the QUESTIONED report as Gerald 
R. McMenamin in CHEVRON CORPORATION v. Steven DONZIGER, et al. written April 3, 2013. Note that these reports are 
practically the same. I have chosen them in this thought experiment to show that, using Dr. McMenamin's forensic stylistics/ 
style markers approach, even vastly similar reports can be analyzed in such a way as to “prove” different authorship. 


53 Let us proceed with the thought experiment. In terms of document formatting, which is claimed in Dr. McMenamin's Dutcher 
v. Bold report to be an indicator of different authorship. Perusing the KNOWN and QUESTIONED reports, I note that the 
KNOWN numbers paragraphs using a single number as in “1.” while the QUESTIONED report uses two numbers as in “2.0”. 
There are numerous examples of this difference, thus presenting a clear indicator of different authorship. 


54 Using this approach, we could select stylistic markers that might show that Dr. McMenamin's own reports were or were 
not written by the same author. The point is that the approach is highly impressionistic and malleable and not a reliable way 
to predict authorship. 


TX. Conclusion and Opinion 


55 The above analytical thought experiment has shown that Dr. McMenamin's forensic stylistic method can easily become a 
tool for advocacy. The methods fails the test of scientific reliability because its methodology lacks scientific rigor. Attempts to 
replicate the research by a second researcher (myself) led to outcomes completely opposite to those found by Dr. McMenamin. 
This is because Dr. McMenamin's research design lacks measures that reduce researcher bias, and, in fact, seem to build in 
measures to ensure researcher bias-it's a feature not a flaw. 


56 Dr. McMenamin's styalistic markers do not appear to have been subjected to any prior scientific or empirical testing. None of 
these style markers has been proven to reliably predict authorship. The markers used in Dr. McMenamin's approach are selected 
on an ad hoc and impressionistic basis without any grounding in empirical, scientific research. Indeed, as demonstrated above, 
the linguistic features or style markers used in Dr. McMenamin's approach can be chosen to support just about any outcome. 


57 Consequently, based upon the previous discussion, it is my opinion that Dr. McMenamin's research methodology as described 
in his Dutcher v. Bold report fails scientific reliability and validity tests. As such, his conclusion that it is highly probable that 
Dan Gilroy did not write the Nightcrawler screenplay lacks scientific credibility. 


Note: The opinions contained herein are stated to a reasonable degree of probability in the field of linguistics. In addition, my 
compensation is not dependent on the outcome or the opinions expressed in this report. 


Footnotes 


1 Ben Zimmer, Decoding Your E-Mail, New York Times, p. SR12 (July 24 2011), at http:/Awww.nytimes.com/2011/07/24/ 
opinion/sunday/24gray.html. 

2 Mark Liberman, High-stakes forensic Linguistics, Language Log (July 25, 2011 8:14 am.), at http:// 
languagelog.ldc.upenn.edu/nll/?p=3309. 
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